Probabilistic Dense Depth from Monocular Images

T his proposal presents a probabilistic formalism that strives for a real time estimation of depth and 3D motion from monocular images. Given a moving camera equipped with an inertial measurement unit and capturing images of a scene with moving objects, we would like to determine, in real time, the 3D structure of the scene in the field of view of the camera as well as the different 3D motions with respect to the camera. Since the structure we are seeking is dense and the frame rate is high it is better to use optical flow rather than point correspondences. This problem has been the subject of thorough study and research over the past two decades; however, in terms of time efficiency, accuracy and ability to deal with diverse situations, the results achieved by various approaches do not meet the requirements of real-life applications. The problem presents many inherent difficulties due mainly to the noise induced by the imaging process, to the optical flow being an approximation of the velocity field and to the sensitivity of the problem to noise implied by the underlying mathematics. Recursive estimation has been always known as a viable framework for this problem. However due to high dimensionality of the depth field, recursive estimation is problematic. Almost all of the previous approaches assume a known motion or estimate depth and motion independently which allows them to do a pixel wise estimation of depth. The proposed formalism is tailored to address these specific problems and rests on three cornerstones. (1) The first one is a Rao-Blackwellised Particle Filter which partitions the state-space into two subspaces, a 3D velocity subspace estimated recursively using a Particle Filter, and a depth subspace conditional on the velocity subspace. The depth field of each particles is cast as a Markov Random Field and estimated in two steps (second and third cornerstones). (2) An Extended Kalman Filter is employed to recursively estimate the depth at each point using optical flow likelihoods. (3) A recursive approach is proposed to segment the depth field and is followed by a probabilistic depth regularization within the segments using belief propagation.